[rustdoc] Correctly handle `should_panic` doctest attribute and fix `--no-run` test flag on the 2024 edition #147674

GuillaumeGomez · 2025-10-14T12:02:10Z

Fixes #143009.
Fixes #143858.

Supersedes #143900.

For --no-run, we forgot to check the "global" options in the 2024 edition, fixed in the first commit.

For should_panic fix, the exit code check has been fixed.

r? @fmease

rustbot · 2025-10-14T12:02:16Z

fmease is currently at their maximum review capacity.
They may take a while to respond.

GuillaumeGomez · 2025-10-14T12:02:47Z

Let's see if it works like that, otherwise I'll add some cfg(bootstrap) around. Locally with stage 2 it worked fine.

GuillaumeGomez · 2025-10-14T15:02:12Z

Ah right, forgot to fix std doc examples.

GuillaumeGomez · 2025-10-15T09:18:52Z

Checking all previously failing CIs.

@bors try jobs=armhf-gnu,test-various,x86_64-gnu-aux

[rustdoc] Correctly handle `should_panic` doctest attribute and fix `--no-run` test flag on the 2024 edition try-job: armhf-gnu try-job: test-various try-job: x86_64-gnu-aux

rust-bors · 2025-10-15T10:25:39Z

💔 Test for 0c093af failed: CI. Failed jobs:

try - test-various (web logs, extended logs)

GuillaumeGomez · 2025-10-16T13:06:33Z

So the error comes from the fact that should_panic is ignored on some platforms by libtest and rustdoc didn't do this check. I added it as well.

Let's try again.

@bors try jobs=armhf-gnu,test-various,x86_64-gnu-aux

[rustdoc] Correctly handle `should_panic` doctest attribute and fix `--no-run` test flag on the 2024 edition try-job: armhf-gnu try-job: test-various try-job: x86_64-gnu-aux

rust-bors · 2025-10-16T14:14:35Z

💔 Test for 7103c01 failed: CI. Failed jobs:

try - test-various (web logs, extended logs)

GuillaumeGomez · 2025-10-16T17:38:43Z

@bors try jobs=armhf-gnu,test-various,x86_64-gnu-aux

[rustdoc] Correctly handle `should_panic` doctest attribute and fix `--no-run` test flag on the 2024 edition try-job: armhf-gnu try-job: test-various try-job: x86_64-gnu-aux

rust-bors · 2025-10-16T18:45:02Z

💔 Test for 38c3e03 failed: CI. Failed jobs:

try - test-various (web logs, extended logs)

GuillaumeGomez · 2025-10-16T19:46:09Z

@bors try jobs=test-various

[rustdoc] Correctly handle `should_panic` doctest attribute and fix `--no-run` test flag on the 2024 edition try-job: test-various

rust-bors · 2025-10-16T20:55:44Z

💔 Test for c5583f8 failed: CI. Failed jobs:

try - test-various (web logs, extended logs)

GuillaumeGomez · 2025-10-17T18:43:09Z

Not ready yet! Just more cleanups before my next attempt: if should_panic (and not compile_fail), then we compile a binary with libtest to ensure we can actually run this test and if so, we call the other code.

GuillaumeGomez · 2025-10-18T00:19:42Z

@bors try jobs=test-various

[rustdoc] Correctly handle `should_panic` doctest attribute and fix `--no-run` test flag on the 2024 edition try-job: test-various

rust-log-analyzer · 2025-10-18T00:57:44Z

The job aarch64-gnu-llvm-20-2 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

---- doctest::tests::comment_in_attrs stdout ----

thread 'doctest::tests::comment_in_attrs' (56156) panicked at src/librustdoc/doctest/tests.rs:419:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\n#![feature(rustdoc_internals)]\n#![allow(internal_features)]\n#![doc(rust_logo)]\n//! This crate has the Rust(tm) branding on it.\npub fn main() {\n\n}", 2)
 right: ("#![allow(unused)]\n#![feature(rustdoc_internals)]\n#![allow(internal_features)]\n#![doc(rust_logo)]\n//! This crate has the Rust(tm) branding on it.\nfn main() {\n\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::comment_in_attrs
   5: <rustdoc::doctest::tests::comment_in_attrs::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::comment_in_attrs stdout end ----
---- doctest::tests::make_test_basic stdout ----

thread 'doctest::tests::make_test_basic' (56158) panicked at src/librustdoc/doctest/tests.rs:53:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\npub fn main() {\nassert_eq!(2+2, 4);\n}", 2)
 right: ("#![allow(unused)]\nfn main() {\nassert_eq!(2+2, 4);\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_basic
   5: <rustdoc::doctest::tests::make_test_basic::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_basic stdout end ----
---- doctest::tests::inner_attributes stdout ----

thread 'doctest::tests::inner_attributes' (56157) panicked at src/librustdoc/doctest/tests.rs:467:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\n\n//! A doc comment that applies to the implicit anonymous module of this crate\n\n\npub fn main() {\npub mod outer_module {\n    //!! - Still an inner line doc (but with a bang at the beginning)\n}\n}", 2)
 right: ("#![allow(unused)]\n\n//! A doc comment that applies to the implicit anonymous module of this crate\n\n\nfn main() {\npub mod outer_module {\n    //!! - Still an inner line doc (but with a bang at the beginning)\n}\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::inner_attributes
   5: <rustdoc::doctest::tests::inner_attributes::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::inner_attributes stdout end ----
---- doctest::tests::make_test_crate_attrs stdout ----

thread 'doctest::tests::make_test_crate_attrs' (56159) panicked at src/librustdoc/doctest/tests.rs:219:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\n#![feature(sick_rad)]\n\npub fn main() {\nassert_eq!(2+2, 4);\n}", 2)
 right: ("#![allow(unused)]\n#![feature(sick_rad)]\n\nfn main() {\nassert_eq!(2+2, 4);\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_crate_attrs
   5: <rustdoc::doctest::tests::make_test_crate_attrs::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_crate_attrs stdout end ----
---- doctest::tests::make_test_crate_name stdout ----

thread 'doctest::tests::make_test_crate_name' (56160) panicked at src/librustdoc/doctest/tests.rs:87:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\npub fn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 3)
 right: ("#![allow(unused)]\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\nfn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 3)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_crate_name
   5: <rustdoc::doctest::tests::make_test_crate_name::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_crate_name stdout end ----
---- doctest::tests::make_test_fake_main stdout ----

thread 'doctest::tests::make_test_fake_main' (56163) panicked at src/librustdoc/doctest/tests.rs:251:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\npub fn main() {\n//Ceci n'est pas une `fn main`\nassert_eq!(2+2, 4);\n}", 2)
 right: ("#![allow(unused)]\nfn main() {\n//Ceci n'est pas une `fn main`\nassert_eq!(2+2, 4);\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_fake_main
   5: <rustdoc::doctest::tests::make_test_fake_main::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_fake_main stdout end ----
---- doctest::tests::make_test_crate_name_no_use stdout ----

thread 'doctest::tests::make_test_crate_name_no_use' (56161) panicked at src/librustdoc/doctest/tests.rs:68:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\npub fn main() {\nassert_eq!(2+2, 4);\n}", 2)
 right: ("#![allow(unused)]\nfn main() {\nassert_eq!(2+2, 4);\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_crate_name_no_use
   5: <rustdoc::doctest::tests::make_test_crate_name_no_use::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_crate_name_no_use stdout end ----
---- doctest::tests::make_test_ignore_std stdout ----

thread 'doctest::tests::make_test_ignore_std' (56164) panicked at src/librustdoc/doctest/tests.rs:122:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\npub fn main() {\nuse std::*;\nassert_eq!(2+2, 4);\n}", 2)
 right: ("#![allow(unused)]\nfn main() {\nuse std::*;\nassert_eq!(2+2, 4);\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_ignore_std
   5: <rustdoc::doctest::tests::make_test_ignore_std::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_ignore_std stdout end ----
---- doctest::tests::make_test_insert_extra_space stdout ----

thread 'doctest::tests::make_test_insert_extra_space' (56165) panicked at src/librustdoc/doctest/tests.rs:374:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\npub fn main() {\n    use std::*;\n    assert_eq!(2+2, 4);\n    eprintln!(\"hello anan\");\n}", 2)
 right: ("#![allow(unused)]\nfn main() {\n    use std::*;\n    assert_eq!(2+2, 4);\n    eprintln!(\"hello anan\");\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_insert_extra_space
   5: <rustdoc::doctest::tests::make_test_insert_extra_space::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_insert_extra_space stdout end ----
---- doctest::tests::make_test_issues_33731 stdout ----

thread 'doctest::tests::make_test_issues_33731' (56168) panicked at src/librustdoc/doctest/tests.rs:303:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\nextern crate hella_qwop;\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\npub fn main() {\nassert_eq!(asdf::foo, 4);\n}", 3)
 right: ("#![allow(unused)]\nextern crate hella_qwop;\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\nfn main() {\nassert_eq!(asdf::foo, 4);\n}", 3)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_issues_33731
   5: <rustdoc::doctest::tests::make_test_issues_33731::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_issues_33731 stdout end ----
---- doctest::tests::make_test_manual_extern_crate stdout ----

thread 'doctest::tests::make_test_manual_extern_crate' (56170) panicked at src/librustdoc/doctest/tests.rs:141:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\nextern crate asdf;\npub fn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2)
 right: ("#![allow(unused)]\nextern crate asdf;\nfn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_manual_extern_crate
   5: <rustdoc::doctest::tests::make_test_manual_extern_crate::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_manual_extern_crate stdout end ----
---- doctest::tests::make_test_issues_21299 stdout ----

thread 'doctest::tests::make_test_issues_21299' (56167) panicked at src/librustdoc/doctest/tests.rs:283:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\npub fn main() {\n// fn main\nassert_eq!(2+2, 4);\n}", 2)
 right: ("#![allow(unused)]\nfn main() {\n// fn main\nassert_eq!(2+2, 4);\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_issues_21299
   5: <rustdoc::doctest::tests::make_test_issues_21299::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_issues_21299 stdout end ----
---- doctest::tests::make_test_manual_extern_crate_with_macro_use stdout ----

thread 'doctest::tests::make_test_manual_extern_crate_with_macro_use' (56171) panicked at src/librustdoc/doctest/tests.rs:158:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\n#[macro_use] extern crate asdf;\npub fn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2)
 right: ("#![allow(unused)]\n#[macro_use] extern crate asdf;\nfn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_manual_extern_crate_with_macro_use
   5: <rustdoc::doctest::tests::make_test_manual_extern_crate_with_macro_use::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_manual_extern_crate_with_macro_use stdout end ----
---- doctest::tests::make_test_named_wrapper stdout ----

thread 'doctest::tests::make_test_named_wrapper' (56172) panicked at src/librustdoc/doctest/tests.rs:355:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\npub fn main() { #[allow(non_snake_case)] fn _doctest_main__some_unique_name() {\nassert_eq!(2+2, 4);\n} _doctest_main__some_unique_name() }", 2)
 right: ("#![allow(unused)]\nfn main() { #[allow(non_snake_case)] fn _doctest_main__some_unique_name() {\nassert_eq!(2+2, 4);\n} _doctest_main__some_unique_name() }", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_named_wrapper
   5: <rustdoc::doctest::tests::make_test_named_wrapper::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_named_wrapper stdout end ----
---- doctest::tests::make_test_opts_attrs stdout ----

thread 'doctest::tests::make_test_opts_attrs' (56174) panicked at src/librustdoc/doctest/tests.rs:178:5:
assertion `left == right` failed
  left: ("#![feature(sick_rad)]\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\npub fn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 3)
 right: ("#![feature(sick_rad)]\n#[allow(unused_extern_crates)]\nextern crate r#asdf;\nfn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 3)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_opts_attrs
   5: <rustdoc::doctest::tests::make_test_opts_attrs::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_opts_attrs stdout end ----
---- doctest::tests::make_test_returns_result stdout ----

thread 'doctest::tests::make_test_returns_result' (56175) panicked at src/librustdoc/doctest/tests.rs:341:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\npub fn main() { fn _inner() -> core::result::Result<(), impl core::fmt::Debug> {\nuse std::io;\nlet mut input = String::new();\nio::stdin().read_line(&mut input)?;\nOk::<(), io:Error>(())\n} _inner().unwrap() }", 2)
 right: ("#![allow(unused)]\nfn main() { fn _inner() -> core::result::Result<(), impl core::fmt::Debug> {\nuse std::io;\nlet mut input = String::new();\nio::stdin().read_line(&mut input)?;\nOk::<(), io:Error>(())\n} _inner().unwrap() }", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_returns_result
   5: <rustdoc::doctest::tests::make_test_returns_result::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_returns_result stdout end ----
---- doctest::tests::make_test_no_crate_inject stdout ----

thread 'doctest::tests::make_test_no_crate_inject' (56173) panicked at src/librustdoc/doctest/tests.rs:104:5:
assertion `left == right` failed
  left: ("#![allow(unused)]\npub fn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2)
 right: ("#![allow(unused)]\nfn main() {\nuse asdf::qwop;\nassert_eq!(2+2, 4);\n}", 2)
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<(alloc::string::String, usize), (alloc::string::String, usize)>
   4: rustdoc::doctest::tests::make_test_no_crate_inject
   5: <rustdoc::doctest::tests::make_test_no_crate_inject::{closure#0} as core::ops::function::FnOnce<()>>::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
---- doctest::tests::make_test_no_crate_inject stdout end ----

rust-log-analyzer · 2025-10-18T01:29:53Z

The job test-various failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

rust-bors · 2025-10-18T01:30:00Z

💔 Test for 8d12b29 failed: CI. Failed jobs:

try - test-various (web logs, extended logs)

GuillaumeGomez added 3 commits October 14, 2025 00:07

Correctly handle --no-run rustdoc test option

507a90b

Fix should_panic on merged doctests

630702b

Add regression tests for no_run and compile_fail

b292354

rustbot assigned fmease Oct 14, 2025

GuillaumeGomez mentioned this pull request Oct 14, 2025

[rustdoc] Correctly handle should_panic doctest attribute and fix --no-run test flag on the 2024 edition #143900

Closed